Papers with language quality
ChatCRS: Incorporating External Knowledge and Goal Guidance for LLM-based Conversational Recommender Systems (2025.findings-naacl)
Copied to clipboard
| Challenge: | Experimental results show that ChatCRS improves language quality and informativeness by 17% and proactivity by 27%. |
| Approach: | They propose a framework to decompose the CRS task into several sub-tasks . they propose 'knowledge retrieval agent' and 'goal-planning agent' |
| Outcome: | The proposed framework improves language quality and informativeness by 17% and proactivity by 27% on two multi-goal CRS datasets. |
Empathetic Persuasion: Reinforcing Empathy and Persuasiveness in Dialogue Systems (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing models for persuasive dialogue lack emotion annotated data, so we use transformers to provide emotion based feedbacks to our RL agent. |
| Approach: | They propose to use a language model to generate empathetic persuasive dialogues . they annotate existing data with emotions and build transformers to provide feedbacks based on emotion. |
| Outcome: | The proposed model increases the rate of generating persuasive responses compared to state-of-the-art models while maintaining the language quality. |
The Interplay of Task Success and Dialogue Quality: An in-depth Evaluation in Task-Oriented Visual Dialogues (2021.eacl-main)
Copied to clipboard
| Challenge: | chit-chat and task-oriented dialogue models are evaluated on their task success metric, but the best model is usually chosen based on task success. |
| Approach: | They compare models playing different games to find out which one is best . they find that this discrepancy is model- and task-agnostic . |
| Outcome: | The proposed model can generate utterances that are indistinguishable from human dialogues by learning to ground, encode, and decode words that do not occur frequently in the training set. |
Context-Aware Language Modeling for Goal-Oriented Dialogue Systems (2022.findings-naacl)
Copied to clipboard
| Challenge: | Goal-oriented dialogue systems face a trade-off between fluent language generation and task-specific control. |
| Approach: | They propose a method to fine-tune language models in a goal-aware way . they evaluate a flight-booking method with a context-assisted language model . |
| Outcome: | The proposed method outperforms the state-of-the-art method on a flight-booking task by 7% in terms of task success. |
Rethinking the Agreement in Human Evaluation Tasks (C18-1)
Copied to clipboard
| Challenge: | In natural language processing, IAA is often viewed as a means of assessing the quality of data on a task, in particular, the reliability. |
| Approach: | They propose a new approach to use agreement metrics in natural language generation evaluation tasks to reduce subjective bias. |
| Outcome: | The proposed approach is based on the inter-annotator agreement (IAA) of natural language generation tasks. |
Polarity Calibration for Opinion Summarization (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing opinions summarization models emphasize the majority opinions while ignoring the minority opinions. |
| Approach: | They propose a method to align output summary and input text to achieve polarity calibration. |
| Outcome: | The proposed model can mitigate the polarity mismatch between output summary and input text, and maintain the content semantic and language quality. |
Differentially Private Language Models for Secure Data Sharing (2022.emnlp-main)
Copied to clipboard
| Challenge: | a variety of deanonymization attacks allow the re-identification of individuals from tabular data. |
| Approach: | They propose to train a language model in a differentially private manner and sample data from it . they find that the model generates fluent textual datasets with privacy guarantees . |
| Outcome: | The proposed methods outperform direct classifiers with DP-SGD in the real-world. |
Learning to Judge: LLMs Designing and Applying Evaluation Rubrics (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language models are increasingly used as evaluators for natural language generation . human rubrics are often static and misaligned with how models internally represent language quality. |
| Approach: | They propose to use large language models to generate interpretable and task-aware evaluation dimensions and apply them within models. |
| Outcome: | The proposed model improves the semantic coherence and scoring reliability of LLM-defined criteria and their alignment with human criteria. |
Generalizable and Explainable Dialogue Generation via Explicit Action Learning (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Conditioned response generation for task-oriented dialogues implicitly optimizes task completion and language quality. |
| Approach: | They propose to learn natural language actions that represent utterances as a span of words. |
| Outcome: | The proposed approach outperforms latent action baselines on a multi-domain dataset. |
GAIfE: Using GenAI to Improve Literacy in Low-resourced Settings (2025.findings-naacl)
Copied to clipboard
Allahsera Auguste Tapo, Nouhoum Coulibaly, Seydou Diallo, Sebastien Diarra, Christopher M Homan, Mamadou K. Keita, Michael Leventhal
| Challenge: | Illiteracy is a predictor of many negative social and personal outcomes in underresourced countries, where few books exist that are suitable for children to learn to read from. |
| Approach: | They propose to use generative AI to create culturally-engaging materials for learning in mali's vehicular language Bambara by multiplying the content by 10 times . authors propose to apply bias-aware tools to reduce illiteracy and improve learning outcomes through native language education. |
| Outcome: | The proposed toolchain and workflow can be adapted to address low literacy in mali using generative AI. |
Is Reference Necessary in the Evaluation of NLG Systems? When and Where? (2024.naacl-long)
Copied to clipboard
| Challenge: | Despite recent advances in reference-free metrics, it has not been well understood when and where they can be used as an alternative to reference-based metrics. |
| Approach: | They propose to use reference-free metrics to evaluate NLG systems . they find they have a higher correlation with human judgment and greater sensitivity to deficiencies in language quality . |
| Outcome: | The proposed metrics exhibit higher correlation with human judgment and greater sensitivity to deficiencies in language quality. |
Math Word Problem Generation with Mathematical Consistency and Problem Context Constraints (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to generate arithmetic math word problems are invalid or have unsatisfactory language quality. |
| Approach: | They propose a method for automatically generating arithmetic math word problems from equations and context. |
| Outcome: | The proposed approach improves language quality and mathematical validity on three real-world MWP datasets. |
REL-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing evaluations of large language models' ability to communicate uncertainty and knowledge limitations focus on the behaviors of their human interlocutors. |
| Approach: | They propose an interaction-centered evaluation approach that quantifies whether and how humans rely on LLMs' responses. |
| Outcome: | The proposed approach quantifies whether and how humans rely on LLMs' responses. |
RADAR: Enhancing Radiology Report Generation with Supplementary Knowledge Injection (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to enhance radiology report generation overlook the knowledge already embedded within the models, leading to redundant information integration. |
| Approach: | They propose a framework for enhancing radiology report generation with supplementary knowledge injection that leverages both internal and external knowledge. |
| Outcome: | Extensive experiments on MIMIC-CXR, CheXpert-Plus, and IU X-ray show that the proposed model outperforms state-of-the-art LLMs in both language quality and clinical accuracy. |